Object Detection based on Combination of Visible and Thermal Videos using A Joint Sample Consensus Background Model

In uncontrolled video surveillance environments, performing efficient foreground segmentation is very challenging. In order to improve robustness and accuracy of object detection, we take advantage of spectral information of both visible and thermal videos. This paper presents a novel joint background model combining visible and thermal videos for foreground object detection in complex scenarios. Different from traditional methods that first detected moving objects in either domain respectively and then fused the detection results, we provide a joint sample consensus background model with four channels (red, green, blue and thermal) to accomplish the object detection and fusion of complementary information simultaneously, which lowers the computational cost of our method. Raw foreground segmentation is obtained in the thermal domain, making initial foreground more accurate. Meantime this can enhance the efficiency of further steps. Time out map (TOM) is utilized to deal with the problem that a newly exposed background is wrongly marked as foreground for a long time. In the updating phase, unlike most sample-based methods using first-in first-out policy, we intentionally employ a random update policy to reserve some older samples. That is, when a pixel is classified as background, we randomly pick up one of the background samples stored for the corresponding pixel to discard. In this manner, the backgrounds, occluded by slow moving foreground or temporally still foreground, can be recovered promptly when they reappear. Experimental results show that the proposed method can achieve accurate and precise detection results.